Double Buffering and Pipelining
Double buffering is a pipelining technique in GPU programming. It allocates two staging buffers and alternates between them, so that the data movement for iteration \(k+1\) can overlap computation for iteration \(k\). Pipelining is the genralized version of double buffering that splits tasks into three stages, and arranges stages to make them overlap to hide latency.
Table of Contents
1. Introduction of Double Buffering
Double buffering (a.k.a. pipelining) is a way to hide latency by using two copies of a buffer. Suppose a task can be split into two stages: load \(\ell_{i}\) and compute \(c_{i}\). What double buffering do is overlapping the load \(\ell_{i}\) of one task and the compute \(c_{j}\) of another task. In detail, when GPU computes on data in buffer A, it simultaneously loads the next chunk of data into buffer B. By doing, the latency introduced by load is hidden behind useful computations. The illustration shows how double buffering works.
